Generic Proxies - Supporting Data Integration Inside the Database

Existing approaches to data integration generally propose building a layer on top of database systems to perform the necessary data transformations and manage data consistency. We show how support for the integration of heterogeneous data sources can instead be built into a database system through the introduction of a generic proxy concept. Over the last two decades a great deal of research in the database and information systems communities has addressed the challenges of data integration. Generally, the problem addressed is how to combine data from different sources to provide a unified user view [1]. Various approaches have been proposed depending on the purpose of the integration and the nature of the data sources, but two broad categories of data integration systems that have received a lot of attention in recent years are mediator [2] and data warehousing [3] systems. These systems tend to have a common architectural approach in that integration is achieved by building extra layers on top of the database systems. We believe that adding internal support for data integration in a database system can have positive effects in the development of data integration systems. In our approach, the integration of external information sources is done using a generic proxy. A generic proxy consists of two parts: the proxy object and the proxy process. The proxy object represents the database view of the external data source. The data from the external source is cached locally, similar to the data warehouse approach. Queries can be executed locally without any communication to the external source. The synchronisation between the database view of the information source and the external information source is done automatically by the database management system in a transparent way. We have defined a proxy programming interface that allows the user to specify how a proxy object interacts with an external source. The user has to write different implementations for different types of external sources. The proxy processes are created from particular implementations of the proxy interface. When a user wants to create a new proxy object, they must specify the name of the proxy and also the list of arguments that are needed in order to initialize the generic proxy. First, a new proxy object is created and stored in the database. Afterwards, the proxy object must be associated with an existing or newly created proxy process. This association is performed using a chain of responsibility approach. All of the existing proxy processes pertaining to the current proxy